Method and a system for determining the location of an object

ABSTRACT

A method for determining the location of a transmitter (respectively a receiver) in a space defined by one or more reflective surfaces, including the steps of sending a signal from the transmitter (respectively from a set of transmitters); receiving by a set of receivers (respectively by a receiver) the transmitted signal and echoes of the transmitted signal reflected by the reflective surfaces; finding by a first computing module the location of the virtual sources (respectively virtual receivers) of the echoes; mirroring by a second computing module the virtual sources (respectively virtual receivers) into the space and obtained mirrored virtual sources (respectively mirrored virtual receivers); combining by a third computing module the mirrored virtual sources (respectively mirrored virtual receivers) so as to obtain location of the transmitter (respectively the receiver). This method makes use of echoes for localizing the source (respectively receiver) when there is no line of sight between the transmitter(s) and the receiver(s).

FIELD OF THE INVENTION

The present invention concerns a method and a system for determining thelocation of an object as a receiver or a transmitter.

DESCRIPTION OF RELATED ART

Indoor localization systems have become a popular research area in thelast years, and many universities and companies do research in thisfield.

There are numerous applications of indoor localization, covering a broadrange of fields. One example is inventory management: keeping track ofthe location and the amount of a particular good in a warehouse. Anotherapplication is object tracking, such as localizing medical personnel orequipment in a hospital or tracking people with limited mobility tounderstand if they need assistance. Yet another application that drewconsiderable attention is location aware services. When you visit amuseum, you are given a device, and if you want to get information abouta specific piece of artwork, you can enter a code related to thatartwork to retrieve it. However, if the device was aware of the locationof the person, it could detect which artwork the person is approachingand information about it can be directly retrieved without the need ofthe user entering any information. Another important fields where indoorlocalization are security and rescue operations, such as detecting thelocation of police dogs trained to find explosives, or localizingfiremen in a building on fire.

Positioning systems, containing both indoor and outdoor localization,can be divided into three main topologies. The first one isself-positioning system, where the receiver receives data fromdistributed transmitters in order determine its own position (e.g. GPS).Second topology is remote positioning, where receivers located atpossibly multiple locations measure the signal coming from the object inorder to localize it. Third topology that has two subcategories iscalled indirect positioning: A data link is used to transfer positioninformation from the self-positioning system to a remote site orvice-versa.

Besides different topologies of indoor localization, there are alsodifferent modalities. One of the modalities is GPS. Although GPS is oneof the most accurate outdoor localization systems, since the signalscoming from the satellites have poor indoor penetration due to shieldingby the concrete, the performance is poor indoors. Another modality isthe radio-frequency identification (RFID). This method has foundapplications in tracking objects in warehouses or assembly lines. Thereare two different RFID setups: passive and active RFID tags. PassiveRFID tags are inexpensive to manufacture, but the reading ranges areshort, typically 1-2 meters, and the tag readers are expensive tomanufacture. Active tags are more expensive to produce, but they haveincreased range and the tag readers are less expensive. Another modalityis using the cellular network. This enables indoor localization when thebuilding is covered by multiple base stations. However, the accuracydepends dramatically on how well the location is covered and on thebuilding's structure. Wireless local area network (WEAN) is also usedfor indoor localization. However, this approach needs preinstalledinfrastructure and the performance fluctuates since the channel changesover time. Another approach is by using ultra-wideband (UWE) radio,which is one of the most promising technologies for indoor localization.First, UWB can send sharp pulses that enable precise localization oftime of arrivals in the receivers. Second, UWB can penetrate walls,equipment and clothing, thus the signal can be observed even behindobstructions. Finally, one can use sound and ultrasound. The equipmentneeded for ultrasound localization is inexpensive and easily accessible.Furthermore, there are planar ultrasound transceivers, that is,transmitters and receivers that emit and receive in the 2D plane. Thisreduced the total amount of reflections we deal with and simplifies thedesign.

Although indoor localization has been studied for twenty years, thefield is still an active research area and open for furtherdevelopments.

It is an aim of the present invention to obviate or mitigate one or moreof the disadvantages of the prior art.

It is another aim of the present invention to find an alternative to thesolutions of the prior art.

BRIEF SUMMARY OF THE INVENTION

According to the invention, these aims are achieved by means of a methodfor determining the location of a transmitter in a space defined by oneor more reflective surfaces, comprising the steps of

sending a signal from this transmitter;

receiving by a set of receivers the transmitted signal and echoes of thetransmitted signal reflected by these reflective surfaces;

finding by a first computing module the location of the virtual sourcesof the echoes;

mirroring by a second computing module the virtual sources into thespace and obtained mirrored virtual sources;

combining by a third computing module the locations of the mirroredvirtual sources so as to obtain the location of the transmitter.

Advantageously the location of the set of receivers and the locationand/or orientation of the reflective surfaces are known.

The method according to the invention locates a source or transmitter ina room with general geometry bounded by reflective surfaces using themeasurements by the set of receivers, e.g. a microphone array. In otherwords, the geometry of the space defined by the reflective surfaces(e.g. a room) does not have to convex, and a direct path between thetransmitter and the receiver(s) is not necessary. The method assumes theknowledge of the room geometry and microphone positions.

The proposed method corresponds to a system architecture fitting inremote localization topology. The method utilizes the direct signal (ifpresent) and early reflections to localize the source. In this contextthe expression “direct signal” indicates the signal sent by transmitterand directly, i.e. without reflections, received by a receiver or theset of receivers.

The method according to the invention advantageously makes use of earlyreflections or echoes, enabling to localize the source even when thereis no line of sight. The method uses method of images to reduce theproblem of indoor source localization to multiple source localization infree field, and then finding the source position. In other words, themethod of images is used to reduce the problem of localizing the sourcein indoor environment to multiple source localization in free space andthen estimating the source position.

In order to localize the multiple virtual sources, combinatoriallyselected echoes from each receiver of the set of receivers are used andthe resulting location is tested to check if the echoes werecorresponding to a single virtual source, for example, the resultinglocation is tested to check if the echoes uniquely localize a singlevirtual source.

After locating the virtual sources, the method according to theinvention finds the position or location inside the room that results inthe generation of such virtual source.

The step of mirroring of the method comprises applying the method ofimages in reverse order (“inverse method of images”, or “inverse imagesource model”). This mirroring procedure could have additionalapplications, whenever it is possible to observe anything throughechoes, and the geometry of the reflective surfaces is known. A possibleapplication is e.g. GPS in urban environments, where the methodaccording to the invention allows to “see” the satellite only through(possibly multiple) reflections.

The method according to the invention localizes an arbitrary number ofimage sources and reflects them iteratively until they are in the room.The main idea is then to use the knowledge of the locations of thevirtual sources for finding the (unknown) location of the real andoriginal source. Advantageously the inverse method of images reflectsthe locations of the virtual sources back to the real source location.

The virtual source could be of arbitrary order (1st, 2nd, 3rd, . . . ).

Advantageously the step of mirroring comprises:

drawing the lines connecting a virtual source to the set of receivers,

finding the reflective surface which intersects these lines,

reflecting the virtual source across this reflective surface andgenerating a reflected virtual source,

storing the points of intersections on the reflective surface,

checking if the reflected virtual source is inside the space defined bythe reflective surfaces,

repeating the previous steps if the reflected virtual source is notinside the space defined by the reflective surfaces.

In other words, if the reflected virtual source is not inside the spacedefined by the reflective surfaces, the following steps are performed:

drawings the lines connecting the stored points of intersections and thereflected virtual source,

finding the new reflective surface which intersects these lines,

reflecting the reflected virtual source across this reflective surfaceand generating a new reflected virtual source,

storing the points of intersections on the new reflective surface,

checking if the new reflected virtual source is inside the space definedby the reflective surfaces,

repeating the previous steps if the new reflected virtual source is notinside the space defined by the reflective surfaces.

In one embodiment, the method according to the invention comprisesechoes' sorting, i.e. grouping the echoes corresponding to a singlevirtual source.

In one embodiment grouping the echoes corresponding to a single virtualsource comprising checking if

$\sum\limits_{i = 1}^{M}\; ( {{{\overset{.}{s} - m_{i}}} - r_{i}} )^{2}$

is less than a threshold, wherein M is the number of receivers, ŝ is theestimated location of the virtual source with the current selection ofechoes, m_(i) is a receiver and r_(i) the distance between thetransmitter and the m_(i) receiver.

Advantageously the method according to the invention comprisesoptimization of the source location. A measure based on the simulatedroom impulse response from the estimated source location is defined tofind the best estimate for the true source position within the reflectedvirtual sources.

After choosing the estimate from the set of reflected virtual sources,its position is optimized based on the difference between simulated andrecorded impulse responses.

In one embodiment the optimization of the source location estimatecomprises a gradient descent method, which is used for optimizing theestimated location based on the simulated room impulse response.

The method according to the invention could comprise the tracking of thetransmitter by using an optimization method. The optimization algorithmcan then be applied to the problem of source tracking, where the newposition of the source is estimated by applying optimization based onthe position estimated in the previous time instance. In other words,the optimization method is applied for tracking a moving source based onprevious position estimates.

By duality, in one particular embodiment, the method can be applied tolocalizing a microphone using multiple transmitters. In anotherparticular embodiment, the method of the invention can be applied toGPS, wherein the transmitter is a mobile device, and the receivers aresatellites (which, although is opposite to the standard whereby mobiledevice are receivers and satellites are the transmitters, isconceptually the same). It will be understood that the method of theinvention can equally be applied to GPS, wherein the transmitter is asatellite, and the receiver is a mobile device. The application of themethod of the invention to GPS will be described in more detail later.

The present invention concerns also a system for determining thelocation of a transmitter, comprising:

one or more reflective surfaces

one transmitter for sending a signal;

a set of receivers for receiving the transmitted signal and echoes ofthe transmitted signal reflected by these reflective surfaces;

a first computing module for finding the location of the virtual sourcesof said echoes;

a second computing module for mirroring the virtual sources into theroom and obtained mirrored virtual sources;

a third computing module for combining the locations of these mirroredvirtual sources so as to find the location of the transmitter.

In one preferred embodiment, the first computing module, the secondcomputing module and the third computing module are the same module.

In one preferred embodiment, the signal is a UWB signal, the transmitterbeing a UWB transmitter and the receivers being UWB receivers. Howeverother kinds of signals (e.g. acoustic, RF, light, etc.) can be used, aswill be discussed.

The present invention concerns also a computer program product,comprising:

a tangible computer usable medium including computer usable program codefor determining the location of a transmitter sending a signal receivedby a set of receivers, the set of receivers receiving also echoes of thetransmitted signal reflected by one or more reflective surfaces, thecomputer usable program code being used for

finding by a first computing module the location of the virtual sourcesof these echoes;

mirroring by a second computing module the virtual sources into thespace and obtained mirrored virtual sources;

combining by a third computing module the locations of these mirroredvirtual sources so as to obtain the location of the transmitter.

By duality, the same procedure can be performed to localize a receiverwith several transmitters in a non-convex room.

The present invention concerns then also a method for determining thelocation of a receiver in a space defined by one or more reflectivesurfaces, comprising the steps of

sending a signal from a set of transmitters;

receiving by this receiver the transmitted signal and echoes of thetransmitted signal reflected by these reflective surfaces;

finding by a first computing module the location of the virtualreceivers of the echoes;

mirroring by a second computing module the virtual receivers into thespace and obtained mirrored virtual receivers;

combining by a third computing module the locations of the mirroredvirtual receivers so as to obtain the location of the receiver.

The present invention concerns then also a system for determining thelocation of a receiver, comprising:

one or more reflective surfaces

a set of transmitters for sending a signal;

this receiver for receiving the transmitted signal and echoes of thetransmitted signal reflected by said reflective surfaces;

a first computing module for finding the location of the virtualreceivers of the echoes;

a second computing module for mirroring the virtual receivers into theroom and obtained mirrored virtual receivers;

a third computing module for combining the locations of the mirroredvirtual receivers so as to find the location of the receiver.

The present invention concerns then also a computer program product,comprising:

a tangible computer usable medium including computer usable program codefor determining the location of a receiver receiving a signaltransmitted by a set of transmitters, the receiver receiving also echoesof the transmitted signal reflected by one or more reflective surfaces,the computer usable program code being used for

finding by a first computing module the location of the virtualreceivers of the echoes;

mirroring by a second computing module the virtual receivers into thespace and obtained mirrored virtual receivers;

combining by a third computing module the locations of the mirroredvirtual receivers so as to obtain the location of the receiver.

Experiments performed by the applicant have demonstrated theeffectiveness, the accuracy and the robustness of the proposed methodsand systems.

The present invention concerns also a computer data carrier storingpresentation content created with the described methods.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be better understood with the aid of the descriptionof an embodiment given by way of example and illustrated by the figures,in which:

FIG. 1A shows an example of trilateration as intersection of circleswithout jitter.

FIG. 1B shows an example of trilateration as intersection of circleswith jitter.

FIG. 2 shows a room comprising a transmitter (source) and a set ofreceivers, the transmitter being localized based on TOA.

FIG. 3 shows a room comprising a transmitter (source) and a set ofreceivers, the transmitter being localized based on TDOA.

FIG. 4 shows a room comprising a transmitter (source) and a set ofreceivers, and an example of virtual sources of the first and secondorder.

FIG. 5A shows a room comprising a transmitter (source), and an exampleof a first order virtual source.

FIG. 5B shows a room comprising a transmitter (source), and an exampleof a first order virtual source, and a second order virtual source.

FIG. 6 illustrates an example of visibility of a second order virtualsource. Although v_(2;4) is visible by the microphone in wall (4), itwill not be heard unless wall (2) extends until point a. With theillustrated room geometry, the microphone can only hear v_(2;4) fromintersection the two illustrated shades in top left corner.

FIG. 7 illustrated an example of labelling obstructive walls by usingconvex hull of the room. Two out of six represented walls are inside theconvex hull of the room, thus it is necessary to check if the signal isblocked by only two walls.

FIG. 8A illustrates the times of arrivals of the echoes coming from thereal source and from the virtual sources of FIG. 8B.

FIG. 8B illustrates a room comprising the real source and a set ofreceivers, and the virtual sources generated from the real source.

FIG. 9 shows an example of how reflecting (mirroring) a localized firstorder virtual source back into a room.

FIGS. 10A to 10C shows an example of three steps for reflecting(mirroring) a localized second order virtual source into the room usingmethod of images in reverse order.

FIG. 11 illustrates recorded RIR by each microphone, and the simulatedRIR from the estimated location.

FIG. 12 illustrates an embodiment of the optimization of positionestimate.

FIG. 13 illustrates the dimensions of the L-shaped room used in sourcelocalization simulations.

FIGS. 14A and 14B show the global view respectively vicinity of the truesource of the simulation results for localization in L-shaped roomwithout measurement jitter.

FIGS. 15A and 15B show the global view respectively vicinity of the truesource of the simulation results for localization in L-shaped room withjitter being i.i.d. centered Gaussian with σ=0.05.

FIGS. 16A and 16B shows the global view respectively vicinity of thetrue source of the simulation results for localization in a room withcomplex geometry, with jitter being i.i.d. centered Gaussian with σ=0.1.

FIG. 17A shows simulation results for tracking (using all of the stepsexcept optimization) a moving source following the path where s₁(t)=1+3cos³(2πt/120) and s₂(t)=6.5+2 sin³(2πt/120) for t=0, 1, 2 . . . 120, thejitter being i.i.d. with distribution ε_(i)˜N(0.0.05).

FIG. 17B shows simulation results for tracking (using only theoptimization based on the previous estimate) a moving source followingthe path where s₁(t)=4+3 cos³(2πt/120) and s₂(t)=6.5+2 sin³(2πt/120) fort=0, 1, 2 . . . 120, the jitter being i.i.d. with distributionε_(i)˜N(0.0.05).

FIGS. 18A to 18D illustrate the contours of formula (11) in 8×8 squareroom for the source position s=(4, 5)^(T), for one microphone (top) andthree microphones (bottom) using direct and first order echoes. Thecontours on FIGS. 18A and 18C are for jitter-free measurements, and thecontours on FIGS. 18B and 18D are for i.i.d centered Gaussian jitterwith

σ=0.05.

FIG. 19 illustrate an example of the l₂ localization error.

FIGS. 20A and 20B illustrate the contours of formula (10) in 8×8 squareroom for the source position s=(4, 5)^(T), for one microphone (FIG. 20A)and three microphones (FIG. 20B) using direct and first order echoes.The contours are plotted for jitter free measurements and the microphonepositions are same with FIGS. 18A to 18D.

FIG. 21 illustrates an embodiment of a system according to theinvention.

FIG. 22 illustrates an embodiment of a data processing system in which amethod in accordance with an embodiment of the present invention may beimplemented.

DETAILED DESCRIPTION OF POSSIBLE EMBODIMENTS OF THE INVENTION

The present invention will be now described in more detail in connectionwith its embodiment for determining the location of a loudspeaker (or,in general, of a transmitter) by knowing the geometry of a room, i.e.the location and the orientation of its walls (or, in general, of itsreflective surfaces), and the location of a set of microphones (or, ingeneral, of a receivers). However the present invention findsapplicability in connection with many other fields, as will bediscussed. For example, the described method and system can be used fordetermining the location of a receiver by knowing the geometry of aroom, i.e. the location and the orientation of its walls or reflectivesurfaces, and the location of a set of transmitters.

The present invention will be now described in more detail in connectionwith an ultrasonic signal. However the present invention findsapplicability of connection with other kinds of signals, e.g. and in anon-limiting way a light signal, an RF signal, etc.

The present invention will be now described in more detail in connectionwith a room. The example described is with respect to a 2D room for easeof understanding of the principles of the present invention; however itwill be understood that the invention can be applied, using the sameprinciples, to a 3D room. It will also be understood that the presentinvention does not necessarily need to be applied in a room.

Notation

Throughout this application vectors are denoted as boldface letters,e.g. x. The position of the i^(th) microphone is denoted as m_(i).Without loss of generality, one corner of the room is assumed to be atthe origin, and the corners are numbered in counter clockwise directionstarting from ‘1’ for the corner at the origin. The coordinates of thei^(th) corner of the room are denoted by c_(i), and the unit outwardnormal vector of the i^(th) wall is denoted by n_(i). The inner productbetween two vectors a and b is denoted as

a, b

=a^(T)b, and ∥a∥ denotes the f₂-norm of a defined as ∥a∥=√{square rootover (

a, a

)}. The coordinates of the source are denoted by sε

². The virtual sources (to be explained) generated by reflecting thesource, in specified order, by the walls, i, j, k is denoted asv_(i,j,k).

DEFINITIONS AND PROPERTIES Times of Arrivals—TOA

Times of arrivals are the measurements of the absolute propagation timefor the signal to reach the microphones after being emitted by thesource. TOA can be measured if there is synchronization between themicrophones and the source, namely, if the microphones have theinformation about when the signal was emitted by the source.

Time Difference of Arrivals—TDOA

If the microphones do not know when the signal is emitted by the source,the absolute propagation time for signal to reach the microphones isunknown, thus TOA are not present. In this case one can measure the timedifference of arrivals between microphones if they have a common timereference. This can be achieved by designating one of the microphones asthe reference microphone, and finding the difference between the timewhen the signal reaches the reference microphone and the othermicrophones.

Direction of Arrival—DOA

Direction of arrival is the angle with respect to a predefinedreference, which the signal is coming to a receiver.

Triangulation

Triangulation is determining the position of an object by measuring theangles between the object and predefined fixed anchors.

Trilateration

Trilateration is a method for finding the position of an object by usingdistance measurements to at least three anchors; in the case where oneuses more than three anchors, the problem is called multilateration.

It should be noted that in TOA, TDOA and DOA preferably at least threereceivers for unique localization in free space in 2D are used, and fourin 3D.

It should be understood that in the present application the terms“virtual source” and “image source” are used interchangeably and bothterms stand for mirror images of a true source across one or multiplewalls. Image sources are used to model echoes. First order echoes can beseen as coming from first order image sources, and higher order echoescan be seen as coming from higher order image sources.

DESCRIPTION OF EMBODIMENTS Localization Based on TOA

If times of arrivals are known, the distance from the source to themicrophones can be found by using the speed of propagation of sound inair C=343.2 m/s (precise value depends on other factors such as thetemperature). Then by trilateration, the position of the source can befound by intersecting the distance circles 1 a-d as shown in FIG. 1 a.

When the time of arrivals are present without measured error, thecircles will intersect at a single point and the intersection gives theposition of the source. However, if there is jitter in the distancemeasurements, the circles 1 a-d do not intersect at a single point asshown in FIG. 1 b. In that case one can pose an optimization problem toestimate the location of the source as is illustrated in FIG. 2.

Referring to FIG. 2, having measured distance r_(i) of the source 2 tothe i^(th) microphone m₁-m₄ (generalized as m_(i)) as:

r _(i) =∥x−m _(i)∥+ε_(i) ,i=1,2, . . . ,M,

where ε_(i) is random measurement jitter and M is the number ofmicrophones, the position of the source 2 can be estimated by findingthe position in the 2D plane that minimizes the sum of the squares ofthe differences between the measured distances r₁,r₂ of the source 2 tothe microphones and the distances between the microphones m₁-m₄(generalized as m_(i)) and the test position 7. This optimizationproblem can be written as:

$\begin{matrix}{\underset{x \in {\mathbb{R}}^{2}}{minimize}{\sum\limits_{i = 1}^{M}\; {( {r_{i} - {{x - m_{i}}}} )^{2}.}}} & (1)\end{matrix}$

The solution to this problem yields the maximum likelihood estimator iferrors follow a Gaussian distribution with covariance matrixproportional to the identity matrix. However, this problem is notconvex, and there is no efficient algorithm to find the globally optimalsolution. There are methods for approximating the solution, such as bysemidefinite relaxation as disclosed in:

-   K. Cheung. W.-K. Ma, and H. So. “Accurate approximation algorithm    for toa-based maximum likelihood mobile location using semidefinite    programming,” in Acoustics, Speech, and Signal Processing. 2004,    Proceedings. (ICASSP '04). IEEE International Conference on, vol. 2,    2004, pp. ii 145 8 vol. 2.    which is incorporated herein with reference.

However, it is reported that although semi-definite relaxation yieldsgood results for some instances, it can perform badly if the relaxationis not tight as disclosed in

-   A. Beck. P. Stoica, and J. Li. “Exact and Approximate Solutions of    Source Localization Problems.” Signal Processing, IEEE Transactions    on, vol. 56, no. 5, pp. 1770 1778, 2008.    which is incorporated herein with reference.

Another optimization problem mentioned is the so called‘squared-range-based least squares’ obtained by squaring the distancesin (1) defined as:

$\begin{matrix}{\underset{x \in {\mathbb{R}}^{2}}{minimize}{\sum\limits_{i = 1}^{M}\; ( {{{x - m_{i}}}^{2} - r_{i}^{2}} )^{2}}} & (2)\end{matrix}$

Although this is still a nonconvex problem, the solution can be foundefficiently and globally by the method described in the publication“Exact and Approximate Solutions of Source Localization Problems”.

To formulate the problem so that the solution can be found efficiently,first, we write it in constrained from as:

$\underset{x \in {\mathbb{R}}^{2}}{minimize}{\sum\limits_{i = 1}^{M}\; ( {\alpha - {2\; m_{i}^{\top}x} + {m_{i}}^{2} - r_{i}^{2}} )^{2}}$subject  to  x² = α.

By using the substitution y=(x^(T),α)^(T) the problem is written as:

$\begin{matrix}{{\underset{y \in {\mathbb{R}}^{3}}{minimize}{{{Ay} - b}}^{2}}{{{{{subject}\mspace{14mu} {to}\mspace{14mu} y^{\top}{Dy}} + {2\; f^{\top}y}} = 0},{where}}{{A = \begin{pmatrix}{{- 2}\; m_{1}^{-}} & 1 \\\vdots & \vdots \\{{- 2}\; m_{M}^{\top}} & 1\end{pmatrix}},{b = \begin{pmatrix}{r_{1}^{2} - {m_{1}}^{2}} & \; \\\vdots & \vdots \\{r_{M}^{2} - {m_{M}}^{2}} & \;\end{pmatrix}},{And},{D = \begin{pmatrix}I_{2} & 0_{2 \times 1} \\0_{1 \times 2} & 0\end{pmatrix}},{f = {\begin{pmatrix}0_{2 \times 1} \\{- 0.5}\end{pmatrix}.}}}} & (3)\end{matrix}$

The resulting problem consists of minimization of a quadratic objectivesubject to a single quadratic equality constraint, which are calledgeneralized trust region sub-problems (GTRS) in optimization literature,as disclosed in:

-   J. J. Moré. “Generalizations of the trust region problem.”    Optimization methods and Software, 1993.    which is incorporated here by reference.

It is shown that yε

³ is an optimal solution if and only if there exists ε such that λε

:

(A ^(T) A+λD)y=A ^(T) b−λf

y ^(T) Dy+2f ^(T) y=0

A ^(T) A+λD≧0.

It follows that the optimal solution to (3) is given by:

{circumflex over (y)}(λ)=(A ^(T) A+λD)^(T)(A ^(T) b−λf),  (4)

Where λ is the unique solution of:

{circumflex over (y)}(D)^(T) Dŷ(λ)−2f ^(T) ŷ(λ)=0  (5)

over the interval

where A^(T)A+λD is positive definite. Interval

satisfying this property can be found by using congruencetransformations. By Sylvester's law of inertia, the matrix C^(T)HC,where C is a non-singular matrix, has the same number of positiveeigenvalues, negative eigenvalues and zero eigenvalue as matrix H, as isdisclosed in:

-   G. Strang. Linear Algebra and Its Applications. 3rd. Brooks Cole.    February 1988.    which is incorporated here by reference.

By saying that H is congruent to G if G=C^(T)HC for some non-singular Cand denoting this equivalence relation by H˜G, we have:

A ^(T) A+λD=(A ^(T) A)^(1/2)(I+λ(A ^(T) A)^(−1/2) D(A ^(T) A)^(−1/2))(A^(T) A)^(1/2) ˜I+λ(A ^(T) A)^(−1/2) D(A ^(T) A)^(−1/2).

since all of the matrices on the right hand side of the equation hasnonnegative eigenvalues, it follows that:

$\mathcal{I} = {( {{- \frac{1}{\lambda_{\max}( {( {A^{\top}A} )^{{- 1}/2}{D( {A^{\top}A} )}^{{- 1}/2}} )}},\infty} ).}$

It is given in the publication “Linear Algebra and Its Applications”that λ can be found by simple bisection algorithm since the function (5)is decreasing on the interval

.

Localization Based on TDOA

Referring to FIG. 3, setting one of the microphones (m₁-m₄) (generalizedas m_(i)) as the reference (denoting the reference microphone as ‘0’)and—without loss of generality—setting it as the origin (0,0), we definethe range difference measurements between each microphone m_(i) andreference microphone 0 as:

d _(i) =∥s−m _(i) ∥−∥s∥,i=1,2, . . . ,M.

Geometrically, the points in the 2D plane that have a fixed distancedifference to two fixed anchors trace a hyperbola. Since distancedifference to two anchors yield one hyperbola, if we have three or moremicrophones, we have multiple hyperbolas and in the presence of preciserange difference measurements, the intersection of the hyperbolas yieldthe source position ‘s’. However, if there is jitter in the measurementsand we have more than two hyperbolas, the hyperbolas will not intersectat a single point. In this case, we can solve an optimization problem tofind the best position estimate for the source position ‘s’. Rewritingthe range difference equality we have:

−2d _(i) ∥s∥−2m _(i) ^(T) s=d _(i) ² −∥m _(i)∥² ,i=1,2, . . . ,M,

which is satisfied in jitter-free measurements.

However when there is jitter, the equality does not hold, but areasonable estimate for the source position ‘s’ can be found by solvingso called squared-range-difference-based least squares problem:

$\begin{matrix}{{\underset{x \in {\mathbb{R}}^{2}}{minimize}{\sum\limits_{i = 1}^{M}\; ( {{{- 2}\; m_{i}^{\top}x} - {2\; d_{i}{x}} - g_{i}} )^{2}}},} & (6)\end{matrix}$

where y_(i)=d_(i) ²−∥m_(i)∥².as is disclosed in

-   J. O. Smith and J. S. Abel. “Closed-form least-squares source    location estimation from range-difference measurements.” Acoustics,    Speech and Signal Processing. IEEE Transactions on, vol. 35, no. 12,    pp. 1661 1669, 1987.    and-   A. Beck, P. Stoica, and J. Li, “Exact and Approximate Solutions of    Source Localization Problems.” Signal Processing. IEEE Transactions    on, vol. 56, no. 5, pp. 1770 1778, 2008.    both of which are incorporated herein by reference.

A closed form solution to this problem is derived in the publication“Exact and Approximate Solutions of Source Localization Problems”.First, the problem is written in constraint form:

$\begin{matrix}{{\underset{y \in {\mathbb{R}}^{3}}{minimize}\mspace{14mu} {{{B\; y} - g}}^{2}}{{{subject}\mspace{14mu} {to}\mspace{14mu} y^{\top}{Cy}} = 0}{{y_{3} \geq 0},{{{where}\mspace{14mu} y} = {\begin{pmatrix}x^{\top} & {x}\end{pmatrix}^{\top}\mspace{14mu} {and}}}}{{B = \begin{pmatrix}{{- 2}m_{1}^{\top}} & {{- 2}d_{1}} \\\vdots & \vdots \\{{- 2}m_{M}^{\top}} & {{- 2}d_{m}}\end{pmatrix}},{C = {\begin{pmatrix}I_{2} & 0_{2 \times 1} \\{0_{1 \times 2} - 1} & \;\end{pmatrix}.}}}} & (7)\end{matrix}$

It is shown in the publication “Exact and Approximate Solutions ofSource Localization Problems” that the sufficient conditions for y to bethe optimal point of the problem is there exists λε

such that:

(B ^(T) B+λC)y=B ^(T) g

B ^(T) B+λC≧0

y ^(T) Cy=0

y ₃≧0.

Using the optimality conditions a procedure prototype is explained tofind optimal solution as:

-   -   1. Find solution λ* to:

y(λ)^(T) Cy(λ)=0,λεI ₁,

Where

y(λ)=(B ^(T) B+λC)⁻¹ B _(T) g,

-   -    and I₁ is the interval where B^(T)B+λC is positive definite.    -   2. If the last entry of y(λ*) is nonnegative, i.e. for z=y(λ*)        we have z₃≧0, then z is the global optimizer of the problem and        position estimate can be found by taking the first two entries        of z.        -   However, it might be the case that the resulting vector does            not satisfy the condition z₃≧0. To find the global optimizer            in this case the necessary optimality conditions derived in            the publication “Exact and Approximate Solutions of Source            Localization Problems” are used, which states that the            optimal solution to (7) is either y=0 or has the form:

t(λ)=(B ^(T) B+λC)⁻¹ B _(T) g,

-   -   -   where λ satisfies:

y(λ)^(T) Cy(λ)=0.

and B^(T)B+λC has at most one negative eigenvalue. The intervals of λcorresponding to these settings can be found by the congruence relationas before as:

B _(T) B−λC=(B ^(T) B)^(1/2)(I+λ(B ^(T) B)^(−1/2) C(B ^(T) B)^(−1/2))(B^(T) B)^(1/2) ˜I+λ(B ^(T) B)^(−1/2) C(B ^(T) B)^(−1/2).

Defining the matrix:

V=(B ^(T) B)^(−1/2) C(B ^(T) B)^(−1/2)

and denoting the i^(th) eigenvalue of V as λ_(i)(V) where theeigenvalues are ordered in decreasing order as λ₁≧λ₂≧λ₃. Since haveB^(T)B positive definite and C has 1 negative and 2 strictly positiveeigenvalues, we have λ₁≧λ₂≧0≧λ₃. From the congruence relation we seethat signs of the eigenvalues of B^(T)B+λC are the same with I+λV whichhas eigenvalues 1+λ·λ_(i)(V). Using these facts there are three disjointintervals where B_(T)B+λC has at most 1 negative eigenvalue:

${1.\mspace{14mu} I_{0}} = {( {{- \frac{1}{\lambda_{3}(V)}},\text{?}} )\text{:}\mspace{14mu} {gives}\mspace{14mu} 2\mspace{14mu} {positive}\mspace{14mu} 1\mspace{14mu} {negative}\mspace{14mu} {eigenvalues}}$${2.\mspace{14mu} I_{1}} = {( {{- \frac{1}{\lambda_{1}(V)}},{- \frac{1}{\lambda \text{?}(V)}}} )\text{:}\mspace{14mu} {gives}\mspace{14mu} 3\mspace{14mu} {positive}\mspace{14mu} {eigenvalues}}$${3.\mspace{14mu} I_{2}} = {( {{- \frac{1}{\lambda_{2}(V)}},{- \frac{1}{\lambda \text{?}(V)}}} )\text{:}\mspace{14mu} {gives}\mspace{14mu} 2\mspace{14mu} {positive}\mspace{14mu} {and}\mspace{14mu} 1\mspace{14mu} {negative}\mspace{14mu} {eigenvalues}}$?indicates text missing or illegible when filed

Using these intervals, the full procedure is defined as:

-   -   1. Find solution λ* to

y(λ)_(T) Cy(λ)=0,λεI ₁.

Where

y(λ)=(B ^(T) B+λC)⁻¹ B ^(T) g.

-   -    If the last entry of y(λ*) is nonnegative, i.e. for z=y(λ*) we        have z₃≧0, then z is the global optimizer of the problem and        position estimate can be found by taking the first two entries        of z. If z₃<0 then perform steps 2 and 3.    -   2. Find all roots λ₁, λ₂, . . . , λ_(p) of

y(λ)^(T) Cy(λ)=0,λεI ₀ ∪I ₂,

-   -    for which third entry of Ills nonnegative.    -   3. Set z as the vector with smallest objective function among 0,        y(λ₁), y(λ₂), . . . , y(λ_(p)). Take position estimate as the        first two entries of z.

Method of Images

Method of images (also known as image source model) provides thatreflections coming from walls can be viewed as direct signals comingfrom virtual sources. These virtual sources are obtained by mirroringthe true source across the reflecting walls (possibly across multiplewalls) as disclosed in:

-   J. Borish. “Extension of the image model to arbitrary polyhedra.”    The Journal of the Acoustical Society of America. 1984.    which is incorporated herein by reference.

FIG. 4 provides an illustration of 1^(st) order virtual source V₁ and V₂(generalized as v₁) generated from walls 5, and 6 (generalized as i)respectively.

The positions of these virtual sources v_(i) can be found by:

$\begin{matrix}\begin{matrix}{v_{i} = {s - {2{n_{i}^{\top}( {s - p_{i}} )}n_{i}}}} \\{= {s - {2{{N_{i}( {s - p_{i}} )}.}}}}\end{matrix} & (8)\end{matrix}$

where

N _(i) :=n _(i) n _(i) ^(T)

is the orthogonal projection operator onto the normal to wall i, andp_(i) is any point belonging to the i^(th) wall.

To find higher order virtual sources one can reflect the source acrossmultiple walls, or equivalently reflect a virtual source across a wall,as:

v _(i,j) =v _(i)−2N(v _(i) −p _(j)).  (9)

By using the method of images, we are reducing the problem of localizingthe source in a room, to localization of multiple sources in free space.

Problem Setup

One of the goals of the present invention is localization of a sourcewhich transmits a signal (e.g. an ultrasonic source or radio source) ina known reverberant room having general geometry, not limited to convex,bounded by at least some planar walls, from the measurements by areceiver (e.g. a microphone array). In the present description anexample in which the source is an ultrasonic source and a receiver is amicrophone array will be described, however it will be understood thatthe present invention is not limited to such an embodiment and othersuitable types of sources and receivers can be used.

When the room is convex, assuming point microphones so they do not blockthe signals, the source is visible by all microphones in the microphonearray (i.e. each microphone in the microphone array can receive a signal(such as an acoustic signal) which is emitted by the source). When thesource is visible, all microphones (receivers) hear the direct signal,and the direct signal arrives before any echo. Thus, in the convex roomsetting, these direct signals can be used for the localization of thesource, and it is reported that the performance of the localizationalgorithms decreases with reverberation, although there are notableexceptions.

In an exemplary problem setting of the present invention, the room isnot assumed to be convex, thus there are positions in the room where thesource is partially visible or not visible by the microphones in themicrophone array (i.e. some microphone in the microphone array cannotdirectly receive a signal (such as an acoustic signal) which is emittedby the source), thus direct signal may not be heard. In the context ofthe present invention a “direct” signal is a signal which has not beenreflected (e.g. which has not been reflected by a wall or object).However, the echoes reflecting from the walls are received by thosemicrophones in the microphone array which do not receive the directsignal. In this setting, echoes are used, which in general makes theperformance worse in the convex room, to localize the source in roomwith general geometry.

With reference to FIGS. 5A and 5B the proposed localization algorithm isas follows. The signal emitted by an ultrasonic source 7 is recordedwith a microphone array. The echoes coming from different walls 5,6 maybe received in different orders at each of the microphones in the array,or put in context of the method of images, signals coming from thevirtual sources V₅ V_(5,6) can be heard (i.e. received) in differentorders by the microphones. Since the virtual source localizationalgorithms explained in above requires distances (or difference ofdistances in TDOA localization) to a single source in order to estimateits position, we need to find which echoes correspond to a singlevirtual source. After finding such echoes and localizing the virtualsources V₅ V_(5,6), we know if they are located inside or outside theroom since the room geometry is known. In case we find that a virtualsource is located a position inside the room we are done. However, if wefind that a virtual source is located a position outside the room, weneed to find the position inside the room that generates that virtualsource, which we do by using the method of images in the reverse order.From the multiple localized and reflected sources, we find the positionthat best estimates the source position and optimize the estimation byusing a measure based on the difference between the recorded andsimulated recordings which will be described in more detail later.

Source Location Estimation

The building blocks of a method according to the present invention willnow be described: the forward model for generating virtual sources givena source position inside the room, localization of virtual sources fromthe recordings by the microphone array, reflecting the localized virtualsources into the room, estimating the source position from multiplereflected sources and optimizing the position of the source locationestimate.

Forward Model

In an embodiment of the present invention a forward model is used whichgenerates the recordings by the microphone array given the roomgeometry, source position and the microphone positions. The approachused for the forward model in this exemplary embodiment is based on themethod disclosed in:

-   J. B. Allen and D. A. Berkley, “Image method for efficiently    simulating small-room acoustics.” The Journal of the Acoustical    Society of America, 1979.    and-   J. Borish, “Extension of the image model to arbitrary polyhedra,”    The Journal of the Acoustical Society of America. 1984.    both of which are incorporated herein by reference.

The positions of the virtual sources can be found by the equationsdisclosed in the previous sections. For the generation of virtualsources and checking if each virtual source is heard by a microphone ina specific position, there are three aspects that need to be tested:validity, visibility and obstruction.

Validity: The virtual source needs to correspond to valid echoes. For acandidate virtual source to be valid, the generating source needs to bedirectly adjacent to that wall. An example of an invalid virtual sourceis reflecting a first generation virtual source back in the room, acrossthe same wall that generated it in the first place. This wouldcorrespond to two consecutive bounces off the same wall, which isphysically infeasible. Visibility: With reference to FIG. 6 in order fora virtual source V₂ V_(2,4) to be heard by a microphone m₁, virtualsource needs to be visible in the wall 4,2 it was generated from. Avirtual source is “visible” in a wall if a line joining the virtualsource and microphone intersects that wall. In other words, the linejoining the virtual V₂ V_(2,4) source and the microphone m₁ shouldintersect the wall 4,2 that generates the virtual source. As can beenseen in FIG. 6 the line joining virtual source V₂ and microphone m₁ doesnot intersect the wall 2 that generates the virtual source V₂, while theline joining virtual source V_(2,4) and microphone m₁ does intersect thewall 4 that generates the virtual source V_(2,4).

Although this is sufficient for a first order echo, for higher orderechoes, one needs to check visibility also in the walls that were usedto generate lower order virtual sources generating it, i.e., point ofintersection ‘a’ of the generating wall and the line drawn between thevirtual source V₂ V_(2,4) and microphone m₁ needs to be visible from theparent walls generating the virtual source. A parent wall is a wall thatis part of the sequence of walls (reflections) that lead to a particularimage source. Visibility of an image source from a certain point meansthat the receiver at that point can hear the signal from the imagesource. Conditions for visibility of higher order image sources are alsoillustrated in FIG. 6. Although V_(2,4) is visible by the microphone inwall (4), it will not be heard by the microphone unless wall (2) extendsto point a. With the illustrated room geometry, the microphone can onlyhear V_(2,4) from the hatched region in the top left corner.

Obstructions: In a convex room, since convex combinations of any set ofpoints belonging to the room is also inside the room there is noobstruction of the source. However, as shown in FIG. 7 when the room isnot convex, there are walls 7,8 that may obstruct the line of sightbetween the microphones and the source. Hence, in non convex room, it isnecessary to check if there is line of sight between the virtual sourceand the microphones. To this end, one can note that only the walls thatviolate the convexity can obstruct the line of sight, thus we may labelthese walls as ‘obstructive’ and check only if they are obstructing theline of sight in each iteration to reduce the number of tests. The‘obstructive’ walls can be found by finding the convex hull of the room,and labelling which walls intersect its interior as shown in FIG. 7.

Localization of Virtual Sources

In this section we discuss the localization of virtual sources in twosettings, where we have TOA and TDOA. First we consider the case wherewe have the signals recorded by M microphones containing the TOA. At thereceiver, we do not know whether the signal is coming directly from thesource or through a reflection from a wall. In particular, if the signalis reflected, we do not know which wall(s) generate the reflection.

In order to localize the virtual sources, we take one arrival time fromeach microphone combinatorially and we calculate the range bymultiplying it with the speed of sound, to obtain the distance r_(i)between the virtual source and microphone where i=1, 2, . . . , M. FIG.8B shows the virtual sources V₁-V₄ generated by real source s and alsoshows microphones mic1-4 FIG. 8A shows the time of arrival atmicrophones mic1-4 of echoes coming from virtual sources V₁-V₄. Fromthese ranges, we localize the position by using squared-range-basedleast squares algorithm to have a position estimate ŝ If the selectedechoes correspond to a correct combination, i.e. they are generated by asingle virtual source, the algorithm will produce a location whosedistances to microphones match r_(i) with high precision. However, ifthe echo combination used for localization does not come from a singlevirtual source, with very high probability, there will be no point inthe 2D plane that will have these distances to the microphones. Based onthis idea we define the localization score, G_(LOC) that measures on howwell the resulting distances and the input distances match as:

${G_{LOC}( \hat{s} )}:={\sum\limits_{i = 1}^{M}\; {( {{{\hat{s} - m_{i}}} - r_{i}} )^{2}.}}$

Using this measure, we say that a particular combination of echoescorresponds to a single virtual source if the score is less than achosen threshold.

In the case where we do not have TOA but TDOA, we use a similar approachto find correct echo combinations corresponding to a single virtualsource. We designate one microphone as the reference microphoneand—without loss of generality—assume it to be at the origin. Then we gothrough each pulse recorded in that reference microphone and wecombinatorially take pulses one from other microphones and multiply thetimes by the speed of sound to obtain distances. Before using the chosenecho combination in the squared-range-difference-based least squaresoptimization, we shift the pulses so that the distance difference in thereference microphone equals to 0 (to have it indeed become thereference). Formally, if t_(i) is the time instances of the selectedpulses from microphones i=0,1, . . . , M−1 where microphone 0 is thereference, we define range-differences as d_(i)=c(t_(i)−t₀), where c isthe speed of propagation of sound. Then we localize the virtual sourceusing squared-range-difference-based least squares algorithm usingdistances d₁, . . . , D_(M-1) and check if the echo combination wascorrect by evaluating range-difference localization score, G_(RDL),defined as:

$\mspace{79mu} {{G_{RDL}(s)}:={\sum\limits_{i = 1}^{M - 1}\; {{( {{{s - m_{i}}} - {\text{?}} - d_{i}} )^{2}.\text{?}}\text{indicates text missing or illegible when filed}}}}$

Again, we accept the chosen echo combination as coming from a singlevirtual source if the score is less than a threshold.

Reflecting Localized Virtual Sources

After finding the location of the virtual source, since the roomgeometry is known, one can use the method of images in reverse order tofind the real source position that would have generated that virtualsource.

Referring to FIG. 9 and FIG. 10A-C, in order to find the real sourcelocation, we draw the lines connecting the virtual source 10 to themicrophones 11 a-d and find the wall 12 that intersects these lines.Then the virtual source is reflected across that wall as shown in FIG.10 b, and the points of intersections on the wall are remembered. If thereflected source is inside the room, as shown in FIG. 9, we are done. Ifnot, as is the case in FIG. 10B, lines connecting the storedintersection points and the new virtual source 15 are drawn, as shown inFIG. 10B, and the virtual source is reflected across the new wall 13 ofintersection, as is illustrated in FIG. 10C. This procedure is iterateduntil position inside room is found, as is shown in FIG. 10C. Thusillustration of this procedure is depicted in FIGS. 10A-C, where asecond order virtual source is reflected into the room.

One problem that might occur while applying the inverse method of imagesis that the lines drawn to multiple microphones may intersect multiplewalls. This may happen due to errors in virtual source localization orjitter. In that case one may choose to drop that localized virtualsource or reflect across the wall with the highest number ofintersections.

Estimating the Source Position

So far we have multiple localized virtual sources that are reflectedinside the room. The remaining questions is how to pick the estimateposition for the true source position. To this end, one may make use ofthe localization scores, G_(LOC) (or G_(RDL) for TDOA), of the virtualsources where the less G_(LOC) is the closer the estimated virtualsource is to the true one. However, as the measurement jitter increases,incorrect echo combinations start to mimic correct echo combinationscoming from false virtual sources and give low localization scores.Hence, for stable localization in case of high measurement jitter, adifferent measure (score) of how good/accurate an position estimate ismay be used:

We derive the score based on the following idea when we have TOArecordings. If the estimated source position is close to the truesource, the simulated room impulse response from the estimated sourcewill be close to the recorded one as shown in FIG. 11. Using this ideawe define a score measuring how close the simulated and the recordedimpulse responses are by taking echo by echo recorded by each microphoneand finding the closest peak in the simulated response. We then sum thesquares of differences between these two. Mathematically, we define theRIR score, G_(RIR), as:

${{G_{RIR}( \hat{s} )}:={\sum\limits_{i = 1}^{M}\; {\sum\limits_{j = 1}^{n_{m}}\; e_{i,j}^{2}}}},{where}$${e_{i,j} = {\min\limits_{k}{{{\hat{r}}_{i,k} - r_{i,j}}}}},$

and r_(i,j) is the j^(th) pulse recorded by i^(th) microphone, and{circumflex over (r)}_(i,k) is the k^(th) pulse that would have beenrecorded by the i^(th) microphone if the source was at ŝ.

Using this measure, we pick the reflected source that gives the leastG_(RIR) as the estimate position.

Optimizing the Position Estimate

Above we have defined a score based on RIR to choose among the localizedand reflected virtual sources the virtual source that accuratelyestimates the true source position. We can further improve the estimateby moving it inside the room so that the RIR score, G_(RIR), isminimized. FIG. 12 provides an illustration depicting the notation ofoptimization of position measurement.

Towards this end, we define the virtual source that simulates the echoclosest in time to the j^(th) echo recorded in i^(th) microphone by{circumflex over (v)}(i,j) as:

${{{\hat{v}}_{({i,j})}( \hat{s} )} = {\underset{k}{\arg \mspace{11mu} \min}{{r_{i,j} - {{{{\hat{\nu}}_{k}( \hat{s} )} - m_{i}}}}}}},$

where we denote virtual sources of any order by single subscript. Withthis notation we define the RIR score again, this time as an explicitfunction of the estimated source position:

${G_{RIR}( \hat{s} )} = {\sum\limits_{i = 1}^{M}\; {\sum\limits_{j}\; {( {r_{i,j} - {{{{\hat{\nu}}_{({i,j})}( \hat{s} )} - m_{i}}}} )^{2}.}}}$

To minimize score one may solve the optimization:

$\begin{matrix}{\mspace{79mu} {\underset{s \in {\text{?}{\mathbb{R}}^{2}}}{minimize}{\sum\limits_{i = 1}^{M}\; {\sum\limits_{j}\; {( {r_{i,j} - {{{{\hat{v}}_{({i,j})}( \hat{s} )} - m_{i}}}} )^{2}.}}}}} & (10) \\{\text{?}\text{indicates text missing or illegible when filed}} & \;\end{matrix}$

Although this problem is again non-convex, if the initial positionestimate is good enough we can find the minimizer iteratively by usinggradient descent, or any other local search technique. The gradient ofthe RIR with respect to the source position is calculated as:

${{\nabla{G_{RIR}( \hat{s} )}} = {\sum\limits_{i = 1}^{M}\; {\sum\limits_{j}\; {2( {\prod\limits_{w \in {\hat{v}\text{?}}}\; ( {I_{2} - {2n_{w}n_{w}^{\top}}} )} )( {{{{{\hat{v}}_{({i,j})}( \hat{s} )} - m_{i}}} - r_{i,j}} )\frac{{{\hat{v}}_{({i,j})}( \hat{s} )} - m_{i}}{{{{\hat{v}}_{({i,j})}( \hat{s} )} - m_{i}}}}}}},{\text{?}\text{indicates text missing or illegible when filed}}$

where the product is over the wall sequence that generates the virtualsource {circumflex over (v)}(i,j). Then the position is optimizediteratively by setting:

ŝ←ŝ−η∇G _(RIR)({circumflex over (s)}),

where η≧0 is the learning rate. The algorithm may be stopped when the l₂norm of the update in the source position is smaller than a predefinedpositive threshold.

Given the measurements with jitter, ε_(i,k) resulting from the virtualsource v_(k) obtained from true source position in microphone i as:

r _(i,k) =∥v _(k) −m _(i)∥+ε_(i,k)

we solve the optimization problem:

$\begin{matrix}{{\underset{x}{minimize}\mspace{14mu} {\sum\limits_{i = 1}^{M}\; {\sum\limits_{k}\; ( {r_{i,k} - {{{\hat{v}}_{k} - m_{i}}}} )^{2}}}},} & (11)\end{matrix}$

equivalently,

$\underset{x}{minimize}\mspace{14mu} {\sum\limits_{i = 1}^{M}\; {\sum\limits_{k}\; {( {{{v_{k} - m_{i}}} + ɛ_{i,k} - {{{\hat{v}}_{k} - m_{i}}}} )^{2}.}}}$

If the jitter is i.i.d. Gaussian, the optimization problem gives thesource position that will generate the echoes with maximum likelihood.Denoting the likelihood of obtaining the set of recorded echoes as:

${{p( {r\hat{s}} )} = {\prod\limits_{i = 1}^{M}\; {\prod\limits_{k}\; {p( {r_{i,k}{\hat{v}}_{k}} )}}}},$

and taking negative logarithm to get the negative log-likelihood, wehave:

$\begin{matrix}{{- {\log( {\prod\limits_{i = 1}^{M}\; {\prod\limits_{k}\; {p( {r_{i,k}{\hat{v}}_{k}} )}}} )}} = {- {\sum\limits_{i = 1}^{M}\; {\sum\limits_{k}\; {\log \; ( {p( {r_{i,k}{\hat{v}}_{k}} )} )}}}}} \\{{= {- {\sum\limits_{i = 1}^{M}\; {\sum\limits_{k}\; \log}}}}\mspace{11mu}} \\{( {\frac{1}{\sqrt{2{\pi\sigma}^{2}}}{\exp ( {{{- ( {r_{i,k} - {{{\hat{v}}_{k} - m_{i}}}} )^{2}}/2}\sigma^{2}} )}} )} \\{{= {\sum\limits_{i = 1}^{M}\; {\sum\limits_{k}\; {{const} \cdot ( {r_{i,k} - {{{\hat{v}}_{k} - m_{i}}}} )^{2}}}}},}\end{matrix}$

where const is a constant that depends on σ. Maximizing the likelihoodis equivalent to minimizing the negative log-likelihood, which yields(11). Hence, by solving (11) we obtain the maximum likelihood estimatorfor the position that generates the set of recorded pulses.

However, since the signals coming from the virtual sources areunlabeled, we do not have direct access to r_(i,k), thus we do not know:

r _(i,k) −∥{circumflex over (v)} _(k) −m _(i)∥.

The minimization problem (10) can be viewed as a heuristic method aimingto solve (11), that estimates

r _(i,k) −∥{circumflex over (v)} _(k) −m _(i)∥

by taking the virtual source that gives the closest time difference tor_(i,k).

Simulations

We will now present simulation results of the present invention forsource localization with TOA measurements. First the results oflocalization in an L-shaped room with and without measurement jitterwill be shown. Then it will be shown how the present invention performsfor room with complex geometry that has no parallel walls and concludeby applying the present invention to tracking a moving source.

It should be noted that in all of the FIGS. 13-21, blank circles denotemicrophone positions and black dots depicts the true source position.Squares are localized and reflected virtual source ordered by theirlocalization score where smaller index denotes better localizationscore. The striped dot marks the reflected virtual source that has thebest RIR score, and the crossed dot is the result of optimizationalgorithm based on the position of striped dot.

Source Localization

For testing the developed indoor localization algorithm, we take atypical nonconvex room having L-shaped geometry shown in FIG. 13.

Referring to FIGS. 14A-16B, to test the performance of the localizationalgorithm without jitter we position the source at (6,7) and use fourmicrophones m₁-m₄ positioned randomly by uniform distribution over thesquare region with bottom left corner at (1,1) and upper right corner at(3,3) to record the signal up to third order echoes. It is important tonote that with this setting, there is no line of sight between thesource and any of the microphones.

An outcome of the localization from jitter-free measurements can be seenin FIGS. 14A,B. It can be seen that the present invention findsestimates close to the true source position, and the localization scoresmark positions close to the true source as begin good. It is also seenthat the RIR score chooses within reflected virtual sources the closestone to the true source position. The crossed dot that depicts theoutcome of the optimization algorithm based on the position of thestriped dot is seen to perfectly localize the source.

FIGS. 15A,B shows an outcome of localization when there is measurementjitter drawn i.i.d. from centered Gaussian with σ=0.05. It is seen thatalthough there are reflected sources in the vicinity of the true sourceposition, the best reflected sources in terms of localization scores areaway from it. Thus picking the reflected source having best localizationscore as the estimate of the true source position is not a valid option.However, also here, the reflected source having the best RIR score isthe one closest to the true source position and the estimate is furtherimproved by applying the optimization step based on that position.

As the last localization simulation we show the result of using thepresent invention for a room with very complex geometry with measurementjitter drawn i.i.d. from centered Gaussian with σ=0.1. As can be seen inFIGS. 16A,B although the reflected sources are distributed in a broadrange, the vicinity of the true source position is still dense.Furthermore, although the positions having best localization scores aredistributed, the RIR score picks the one that is closest to the truesource position and optimization algorithm gives an even closerestimate.

Source Tracking

One approach for source tracking is by taking measurements at distincttime instances and localizing the position independently of the previousones. However, since the position of the source depends on its history,one can leverage the previous positions estimates in localizing thesource.

In this simulation we compare the performance of tracking a movingsource with two approaches. First approach is by going through all ofthe steps of the algorithm by recording the signal, finding echoescorresponding to virtual sources and localizing them, reflecting thelocalized virtual sources and taking the one giving the highest RIRscore but not applying the optimization algorithm. The second method isby localizing first position by using first method and in additionapplying optimization algorithm and for other time instances, applyingonly optimization algorithm based on the position estimate of theprevious time instance.

FIGS. 17A,B show the results of the two approaches where the sourcetraces the curve:

sε

²,

where,

s ₁(t)=4−3 cos³(2πt/120)

and s₂(t)=6.5+2 sin³(2πt/120) for t=0, 1, 2, . . . , 120, and the jitteris drawn i.i.d. from centered Gaussian with σ=0.05.

FIG. 17A shows the result of the first approach and FIG. 17B shows theresult of the second approach. As can be seen the second approachoutperforms the first approach while being computationally lighter.

We will now discuss the objective functions behind the optimization stepand plot average localization error for special case of square roomthrough simulations.

FIG. 18A-D show the contours of the function (11) which the presentinvention aims to minimize by solving the heuristic optimization (10).The contours are drawn for a square room of size 8×8 for the sourceposition of:

s=(4,5)^(T)

and first order virtual sources generated from this position.

FIGS. 18A and 18B show the contours for one microphone for measurementswithout jitter (FIG. 18A) and with jitter (FIG. 18B). As can be seen,when there is no jitter in the measurements, the contours are smooth andthe optimal position can be found, e.g., by gradient descent algorithm.With jitter, the contours gets distorted and local minima can occur,hence gradient descent algorithm may get stuck in local minima. However,as seen from FIGS. 18A and 18B as the number of microphones increase thecost curves becomes smoother with for same jitter level.

FIG. 19 shows the average l₂ localization error defined as the Euclideandistance between the true source and the estimated source positions,i.e.

E _(l) ₂ :=∥s−ŝ∥.

In order to find the optimal position, the algorithm is started from thevicinity of true source position and gradient descent algorithm is used.The plot shows that localization error based on minimization of (11)increases smoothly as the jitter is increased.

However, as explained in earlier above, since we do not have the labelsfor the echoes, we cannot minimize (11) so we estimate its solution byusing the heuristic method (10). The contours for the (10) are plottedin FIG. 10A for jitter free measurements using one microphone, and areplotted in FIG. 10E for jitter free measurements using threemicrophones. As can be seen, the resulting function has local minima,hence gradient descent algorithm may get stuck at local minima. However,as also can be seen by comparing the two plots, as the number ofmicrophones increases the contours become smoother, and if the startingpoint of the algorithm is close enough to the optimal position, one canfind the global minimizer by the gradient descent algorithm.

FURTHER ASPECTS OF THE PRESENT INVENTION

Finally, we will now describe FIG. 21 which illustrates an embodiment ofa system according to the invention and FIG. 22 which illustrates anembodiment of a data processing system in which a method in accordancewith an embodiment of the present invention may be implemented.

FIG. 22 is an embodiment of a data processing system 300 in which anembodiment of a method of the present invention may be implemented. Thedata processing system 300 of FIG. 22 may be located and/or otherwiseoperate at any node of a computer network, that may exemplarily compriseclients, servers, etc., and it is not illustrated in the Figure. In theembodiment illustrated in FIG. 22, data processing system 300 includescommunications fabric 302, which provides communications betweenprocessor unit 304, memory 306, persistent storage 308, communicationsunit 310, input/output (I/O) unit 312, and display 314.

Processor unit 304 serves to execute instructions for software that maybe loaded into memory 306. Processor unit 304 may be a set of one ormore processors or may be a multi-processor core, depending on theparticular implementation. Further, processor unit 304 may beimplemented using one or more heterogeneous processor systems in which amain processor is present with secondary processors on a single chip. Asanother illustrative example, the processor unit 304 may be a symmetricmulti-processor system containing multiple processors of the same type.

In some embodiments, the memory 306 shown in FIG. 22 may be a randomaccess memory or any other suitable volatile or non-volatile storagedevice. The persistent storage 308 may take various forms depending onthe particular implementation. For example, the persistent storage 308may contain one or more components or devices. The persistent storage308 may be a hard drive, a flash memory, a rewritable optical disk, arewritable magnetic tape, or some combination of the above. The mediaused by the persistent storage 308 also may be removable such as, butnot limited to, a removable hard drive.

The communications unit 310 shown in FIG. 22 provides for communicationswith other data processing systems or devices. In these examples,communications unit 310 is a network interface card. Modems, cable modemand Ethernet cards are just a few of the currently available types ofnetwork interface adapters. Communications unit 310 may providecommunications through the use of either or both physical and wirelesscommunications links.

The input/output unit 312 shown in FIG. 22 enables input and output ofdata with other devices that may be connected to data processing system300. In some embodiments, input/output unit 312 may provide a connectionfor user input through a keyboard and mouse. Further, input/output unit312 may send output to a printer. Display 314 provides a mechanism todisplay information to a user.

Instructions for the operating system and applications or programs arelocated on the persistent storage 308. These instructions may be loadedinto the memory 306 for execution by processor unit 304. The processesof the different embodiments may be performed by processor unit 304using computer implemented instructions, which may be located in amemory, such as memory 306. These instructions are referred to asprogram code, computer usable program code, or computer readable programcode that may be read and executed by a processor in processor unit 304.The program code in the different embodiments may be embodied ondifferent physical or tangible computer readable media, such as memory306 or persistent storage 308.

Program code 316 is located in a functional form on the computerreadable media 318 that is selectively removable and may be loaded ontoor transferred to data processing system 300 for execution by processorunit 304. Program code 316 and computer readable media 318 form acomputer program product 320 in these examples. In one example, thecomputer readable media 318 may be in a tangible form, such as, forexample, an optical or magnetic disc that is inserted or placed into adrive or other device that is part of persistent storage 308 fortransfer onto a storage device, such as a hard drive that is part ofpersistent storage 308. In a tangible form, the computer readable media318 also may take the form of a persistent storage, such as a harddrive, a thumb drive, or a flash memory that is connected to dataprocessing system 300. The tangible form of computer readable media 318is also referred to as computer recordable storage media. In someinstances, computer readable media 318 may not be removable.

Alternatively, the program code 316 may be transferred to dataprocessing system 300 from computer readable media 318 through acommunications link to communications unit 310 and/or through aconnection to input/output unit 312. The communications link and/or theconnection may be physical or wireless in the illustrative examples. Thecomputer readable media also may take the form of non-tangible media,such as communications links or wireless transmissions containing theprogram code.

The different components illustrated for data processing system 300 arenot meant to provide architectural limitations to the manner in whichdifferent embodiments may be implemented. The different illustrativeembodiments may be implemented in a data processing system includingcomponents in addition to or in place of those illustrated for dataprocessing system 300. Other components shown in FIG. 22 can be variedfrom the illustrative examples shown. For example, a storage device indata processing system 300 is any hardware apparatus that may storedata. Memory 306, persistent storage 308, and computer readable media318 are examples of storage devices in a tangible form.

Therefore, as explained at least in connection with FIG. 22 the presentinvention is as well directed to a system for determining location of anobject, a computer program product for determining location of an objectand a computer data carrier.

In accordance with a further embodiment of the present invention isprovided for a computer data carrier storing presentation contentcreated while employing the methods of the present invention.

Although the present invention has been described in more detail inconnection with its embodiment for determining the location of aloudspeaker or a microphone, the present invention finds applicabilityof connection with many other fields.

The present invention can be used for determining the exact position ofa receiver r, which is a person in the FIG. 21. In the case a satellite,e.g. a GPS satellite is the source s of a radio signal which can bereflected by some buildings B1, B2. If the echo e1 is not used, thelocalisation of a mobile device r of a person can be computedincorrectly (the mobile device r will be considered located incorrespondence of {tilde over (e)}).

Knowing the position of the satellite s, the position of the buildingsB1, B2, etc. (this is possible e.g. by using an electronic map) andapplying the method according to the invention, it is possible toaccurately locate the mobile device r and then the person, without anyerror.

1. A method for determining the location of a transmitter in a spacedefined by one or more reflective surfaces, comprising the steps ofsending a signal from said transmitter; receiving by a set of receiversthe transmitted signal and echoes of the transmitted signal reflected bysaid reflective surfaces; finding by a first computing module thelocation of the virtual sources of said echoes; mirroring by a secondcomputing module the virtual sources into the space and obtainingmirrored virtual sources; combining by a third computing module thelocations of said mirrored virtual sources so as to obtain the locationof the transmitter.
 2. The method of claim 1, the location of the set ofreceivers and the location and/or orientation of the reflective surfacesbeing known.
 3. The method of claim 1, the step of mirroring comprisingapplying the method of images in reverse order.
 4. The method of claim1, the step of mirroring comprising: drawing the lines connecting avirtual source to the set of receivers, finding the reflective surfacewhich intersects said lines, reflecting the virtual source across saidreflective surface and generating a reflected virtual source, storingthe points of intersections on the reflective surface, checking if thereflected virtual source is inside the space defined by the reflectivesurfaces, repeating the previous steps if the reflected virtual sourceis not inside the space defined by the reflective surfaces.
 5. Themethod of claim 1, comprising, if the reflected virtual source is notinside the space defined by the reflective surfaces, drawing the linesconnecting the stored points of intersections and the reflected virtualsource, finding the new reflective surface which intersects said lines,reflecting the reflected virtual source across said reflective surfaceand generating a new reflected virtual source, storing the points ofintersections on the new reflective surface, checking if the newreflected virtual source is inside the space defined by the reflectivesurfaces, repeating the previous steps if the new reflected virtualsource is not inside the space defined by the reflective surfaces. 6.The method of claim 1, comprising echoes' sorting.
 7. The method ofclaim 6, said echo sorting comprising grouping the echoes correspondingto a single virtual source.
 8. The method of claim 7, wherein groupingthe echoes corresponding to a single virtual source comprising checkingif$\sum\limits_{i = 1}^{M}\; ( {{{s - m_{i}}} - r_{i}} )^{2}$is less than a threshold, wherein M is the number of receivers, ŝ is thevirtual source, m_(i) is a receiver and r_(i) the distance between thetransmitter and the m_(i) receiver.
 9. The method of claim 1, comprisingthe optimization of the location of the transmitter.
 10. The method ofclaim 9, said optimization comprising a gradient descent method.
 11. Themethod of claim 10, comprising the tracking of the transmitter by usingsaid optimization method, e.g. by using the gradient descent method. 12.The method of claim 1, the reflective surfaces defining a non-convexspace.
 13. The method of claim 1, the transmitter being a mobile device,the receivers being satellites.
 14. A system for determining thelocation of a transmitter, comprising: one or more reflective surfacesone transmitter for sending a signal; a set of receivers for receivingthe transmitted signal and echoes of the transmitted signal reflected bysaid reflective surfaces; a first computing module for finding thelocation of the virtual sources of said echoes; a second computingmodule for mirroring the virtual sources into the room and obtainedmirrored virtual sources; a third computing module for combining thelocations of said mirrored virtual sources so as to find the location ofthe transmitter.
 15. The system of claim 14, the first computing module,the second computing module and the third computing module being thesame module.
 16. The system of claim 14, the reflective surfacesdefining a non-convex space.
 17. The system of claim 14, the transmitterbeing a mobile device, the receivers being satellites.
 18. The system ofclaim 14, the signal being a UWB signal, the transmitter being a UWBtransmitter, and the receivers being an array of UWB receivers.
 19. Acomputer program product, comprising: a tangible computer usable mediumincluding computer usable program code for determining the location of atransmitter sending a signal received by a set of receivers, the set ofreceivers receiving also echoes of the transmitted signal reflected byone or more reflective surfaces, the computer usable program code beingused for finding by a first computing module the location of the virtualsources of said echoes; mirroring by a second computing module thevirtual sources into the space and obtained mirrored virtual sources;combining by a third computing module the locations of said mirroredvirtual sources so as to obtain the location of the transmitter.
 20. Amethod for determining the location of a receiver in a space defined byone or more reflective surfaces, comprising the steps of sending asignal from a set of transmitters; receiving by said receiver thetransmitted signal and echoes of the transmitted signal reflected bysaid reflective surfaces; finding by a first computing module thelocation of the virtual receivers of said echoes; mirroring by a secondcomputing module the virtual receivers into the space and obtainedmirrored virtual receivers; combining by a third computing module thelocations of said mirrored virtual receivers so as to obtain thelocation of the receiver.
 21. The method of claim 20, the location ofthe set of transmitters and the location and/or orientation of thereflective surfaces being known.
 22. The method of claim 20, the step ofmirroring comprising applying the method of images in reverse order. 23.The method of claim 20, the step of mirroring comprising: drawing thelines connecting a virtual receiver to the set of transmitters, findingthe reflective surface which intersects said lines, reflecting thevirtual receiver across said reflective surface and generating areflected virtual receiver, storing the points of intersections on thereflective surface, checking if the reflected virtual receiver is insidethe space defined by the reflective surfaces, repeating the previoussteps if the reflected virtual receiver is not inside the space definedby the reflective surfaces.
 24. The method of claim 20, comprising, ifthe reflected virtual receiver is not inside the space defined by thereflective surfaces, drawings the lines connecting the stored points ofintersections and the reflected virtual receiver, finding the newreflective surface which intersects said lines, reflecting the reflectedvirtual receiver across said reflective surface and generating a newreflected virtual receiver, storing the points of intersections on thenew reflective surface, checking if the new reflected virtual receiveris inside the space defined by the reflective surfaces, repeating theprevious steps if the new reflected virtual receiver is not inside thespace defined by the reflective surfaces.
 25. The method of claim 20,the mirroring comprising echoes' sorting.
 26. The method of claim 20,comprising the optimization of the position estimate.
 27. The method ofclaim 26, said optimization of the position estimate comprising agradient descent method.
 28. The method of claim 27, comprising thetracking of the receiver by using the optimization method, e.g. thegradient descent method.
 29. The method of claim 20, the reflectivesurfaces defining a non-convex space.
 30. A system for determining thelocation of a receiver, comprising: one or more reflective surfaces aset of transmitters for sending a signal; said receiver for receivingthe transmitted signal and echoes of the transmitted signal reflected bysaid reflective surfaces; a first computing module for finding thelocation of the virtual receivers of said echoes; a second computingmodule for mirroring the virtual receivers into the room and obtainedmirrored virtual receivers; a third computing module for combining thelocations of said mirrored virtual receivers so as to find the locationof the receiver.
 31. The system of claim 30, the first computing module,the second computing module and the third computing module being thesame module.
 32. The system of claim 30, the reflective surfacesdefining a non-convex space.
 33. The system of claim 30, the receiverbeing a mobile device, the transmitters being satellites.
 34. The systemof claim 30, the signal being a UWB signal, the transmitters being anarray of UWB transmitters and the receiver being a UWB receiver.
 35. Acomputer program product, comprising: a tangible computer usable mediumincluding computer usable program code for determining the location of areceiver receiving a signal transmitted by a set of transmitters, thereceiver receiving also echoes of the transmitted signal reflected byone or more reflective surfaces, the computer usable program code beingused for finding by a first computing module the location of the virtualreceivers of said echoes; mirroring by a second computing module thevirtual receivers into the space and obtained mirrored virtualreceivers; combining by a third computing module the locations of saidmirrored virtual receivers so as to obtain the location of the receiver.